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ABSTRACT 

This study explored the mental components that are most 
important for successful problem solving. Four components were studied: 
cognition, metacognition, noncognitive variables, and justification skills. 

It was hypothesized that cognition and knowledge of cognition would predict 
success at solving well-structured problems, which have one correct answer 
and typically only one way to reach a final solution. It was further 
hypothesized that regulation of cognition, justification skills, and 
noncognitive variables, in addition to cognition and knowledge of cognition, 
would predict success at solving ill-structured problems, which have no 
correct answers and multiple ways to reach a final solution. The first study 
included 118 ninth graders from a Midwestern high school and the second study 
included 1,160 middle school students. Results of the two studies support the 
hypothesis that mental components used in problem solving depend on the 
nature of problem solving tasks. Different problem solving tasks require 
different mental components. To promote students* problem solving skills, 
educators must develop teaching and learning strategies that use different 
cognitive components. More research, using appropriate instruments, should 
explore the role of justification skills in solving problems. (Contains 2 
tables and 25 -.ref erences . ) (SLD) 
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The purpose of this study was to explore which mental components are most 
important for successful problem solving. Researchers have argued that problem .solving 
is a complex skill that is influenced by a set of mental components namely, cognition, 
metacognition, non-cognitive variables and justification skills (Jonassen, 1 997; Sinnotte, 

1 989; Vo.ss, 1 988). The extent to which these related components contribute towards 
successful problem solving depends on the nature of the task (Brabeck & Wood, 1 990). 
Using regre,ssion analysis we sought to determine which mental components are 
necessary for various problem-solving tasks. 

For this study we investigated four mental components: cognition, metacognition, 
non-cognitive variables, and Justification skills. Cognition helps learners find an 
appropriate solution from memory (Glaser, 1 989). Metacognition helps learners 
understand and regulate their performance during problem solving. Knowledge of 
cognition is one part of metacognition. It is necessary when solvers do not have 
appropriate solutions in memory. With knowledge of cognition, students can search for 
general strategies that may be used to solve the problem. Regulation of cognition such as 
monitoring, evaluating, and planning is another part of metacognition. It is required when 
learners need to solve problems that have no clear solutions and require large amounts of 
information in various content areas (Kluwe & Friedrick.sen, 1 985; Rainer & Gunnar, 

1 985). Non-cognitive variables (e.g., affect, value, motivation, emotionality, and attitude) 
keep learners going and motivate them to continue through the process (Jonassen, 1 997; 
Sinnott, 1 989). Finally, Justification skills are necessary when learners have to solve 
problems that have commonly divergent or alternative solutions (Voss, 1 988; Voss & 
Post, 1 989). They allow students to develop a rationale for the solution and defend it 
against alternatives. 

We hypothesized that cognition and knowledge of cognition would predict 
success at solving well-structured problems, which have one correct answer and typically 
only one way to reach a final solution (Simon, 1 978 & 1979; Bransford & Stein, 1984; 
Newell & Simon, 1972). On the other hand, we hypothesized that regulation of 
cognition. Justification skills, and non-cognitive variables, in addition to cognition and 
knowledge of cognition, would predict success at solving ill-structured problems, which 
have no correct answers and multiple ways to reach a final solution (Voss, 1 988; 

Sinnotte, 1989). 



METHOD 



Participants/Procedure 

Study 1 included 9"' grade students (n = 118) from a Midwestern high school. 
Students used the high school version of Astronomy Village®, which focuses on stars and 
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stellar evolution. They conducted one investigation over a Four- week period. The 
measures of problem solving and mental components related to this study were 
administered as posttest only. 

Study 2 included grade students (n = 1 160) from a variety of middle 
schools throughout the United States. These students used the middle-school version of 
Astronomy Village, which focuses on solar system content. Students conducted anywhere 
from 2-4 investigations over a four-week period. The measures of problem solving and 
mental components related to this study were administered both pre and post. (For a more 
detailed description of the two Astronomy Village programs refer to Pompea & Blurton, 
1995; McGee & Howard, 1999). 

Measures of Problem-Solving (Dependent Measures). 

Study I used an open-ended response format to present students with both well- 
structured and ill-structured problems. There were two well-structured problems. One 
asked students to describe how they would find the distance to a nearby star. The other 
asked how they would use variable stars to determine the distance to a nearby galaxy. 
These problems were developed based on the content covered in Astronomy Village and 
required students to explain the approach they took to reach a final solution. Both well- 
structured problems have a correct answer. 

There were also two ill-structured problems. One asked students to describe how 
they would form a team to investigate whether or not an asteroid is on a collision course 
with Earth. This one was considered less-structured since it was an extension of ideas 
presented in Astronomy Village. The second asked students to select the site for a new 
telescope from a list of three alternatives, none of which is optimal. Since this problem 
was not related to the content in Astronomy Village it was considered ill-structured. Both 
ill-structured problems allow for multiple possible answers. These open-ended response 
items have been validated in a previous study (Hong, 1998). 

Study 2 used a multiple-choice format to measure problem solving skills. We 
decided to use a multiple-choice format for middle-school students, rather than an open- 
ended response format, because we felt that the writing ability at that age level would not 
allow for accurate assessment of problem solving. There were 26 multiple choice items 
that focused on students’ ability to draw conclusions from data and to infer planetary 
processes from planetary images. For example, students may be presented with a variety 
of data about an imaginary planet’s surface features and asked to determine whether it 
would be fruitful to search for life on that planet. The validation of this assessment 
instrument is in progress and will be reported in the final paper. 

Measures of Mental Components (Independent Measures). 

In Study I, students’ cognition was measured by asking students to classify 
important concepts related to a given problem. In addition, they had to describe the 
relationships between the selected concepts (Clark, 1990; Jonassen, Beisser, & Yacci, 
1993). Metacognition was measured using the How Do You Solve Problems (HSP) 
inventory (see Fortunato, Hecht, Tittle, & Alvarex 1991). HSP is a 21 -item likert scale 
inventory that measures those aspects of metacognition in which the subject calls upon 
knowledge of cognition and regulation of cognition. Justification skills were measured by 
open-ended essay questions that asked students to justify their final solutions. In Study 2, 
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cognition was measured using multiple choice questions. The validation of this 
instrument is in progress and will be reported in the final paper. Metacognition was 
measured using the Inventory of Metacognitive Self-Regulation (IMSR) developed at the 
Center for Educational Technologies™. See Howard, McGee, Hong, & Shia (2000) for 
the validation of this instrument. It measures both knowledge of cognition and regulation 
of cognition. Study 2 did not have a mea.sure of justification skills. Both Study I and 
Study 2 used the Test of Science Related Attitudes as a measure of non-cognitive skills, 
values, attitudes, and beliefs toward science (Fraser, l978;Smist, Archambault, & Owen, 
1994). 



RESULTS 

Simultaneous regression analytic techniques were u.sed to test which components 
among all those entered in the analysis were statistically significant predictors of different 
degrees of structured problem-solving scores. The overall results from the regression 
analysis can be summarized by the following chart. 



Variables 
Kinds of Problems 


R“ 


Cognition 


Justification 

Skills 


Knowledge 
of Cognition 


Regulation 
of Cognition 


Non-Cognitive 

Variables 


MC Well-Slruclured 


57%** 


(3 = 73:1.* 


N/A 


P =.09** 






OE Well-Slriictured 


47%** 


(3 = 38** 


(3=43:1=* 








OE Less- Structured 


6 1 %** 


(3 =.28* 


(3 = 46** 






P =15* 


OE Ill-Structured 


48%** 


P=.48** 


P=.32** 




p =.20* 





Note. MC: Muliiple Clioice.s. OE: Open-ended. = p < .000. * = p<.01. 

P indicaie.s a .signit'icanl prediclor. 



The results of the present study support the hypothe.sis that mental components 
u.sed in problem solving depend on the nature of the problem solving tasks. Overall, 
cognition was a powerful predictor for all degrees of structured problem-solving scores. 
The results support the theory that students who po.s.sess an appropriate, well-organized 
knowledge base are able to solve problems directly because they recognize each problem 
from previous experience and know which moves are appropriate (Chi, et al., 1981; 
Gla.ser, 1984; Resnick, 1983). It was hypothesized that Justification skills would be an 
important component for .solving ill-structured problems but not well-structured 
problems. However, it turned out to be an important predictor for all open-ended 
problem-.solving scores. The results indicate that if students can provide logical 
arguments to support their opinion in a given situation, they may successfully solve an 
open-ended problem whether it is well- or ill-structured (Voss, 1988; Voss & Post, 1989; 
Jonas.sen, 1997). 

Knowledge of cognition is a strong predictor of multiple choice, well-structured 
problem-solving scores. It supports the theory that students require the use of general 
.searching strategies when they do not pos.ses previous experience with a specific type of 
problem (Chi, Bassok, Lewis, Reimann, & Gla.ser, 1988; Gick, 1986). However, the 
results did not show that knowledge of cognition is a predictor in open-ended problem 
solving. It indirectly showed that knowledge of cognition may not be sufficient for 
finding a solution for open-ended problems. Alternatively, regulation of cognition was a 
strong predictor in .solving only open-ended ill-structured problems. It is consistent with 
the theory. that students need to u.se regulation of cognition to keep track of the solution 
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activity and the effects of their efforts because of the uncertain ill-structured problem 
situation (Kluwe & Friedrichsen, 1985; Herbert & Diome, 1993; Rebok, 1989). The 
results suggest that problems have to be complicated enough to challenge students to use 
regulation of cognition for reaching successful solution. In other words, students may not 
need to use regulation of cognition if the problems lack conceptual and structural 
complexity, even though they have those skills, 

Non-cognitive variables were indicated as an important component for solving an 
open-ended, less-structured problem but not an ill-structured problem (Sinnott, 1989; 
Sheurman, 1995), A potential reason for the non-significance in the case of the ill- 
structured problem may be due to the nature of the problem task. In the ill-structured 
problem students had to consider multiple perspectives such as budget, human life, and 
geography, which were not directly related to science, in order to reach a successful 
solution. Although students may not have a strong positive attitude, value, and belief 
toward science, they may be encouraged to continue solving the problem by an internal 
affective reward from the other issues in the problem, 

CONCULSIONS 

The results of this study have important implications for instructional practice, 
especially for science education in a multimedia learning environment. The overall 
results of the investigation illustrate that different problem solving tasks require different 
mental components. Therefore, in order to promote students’ problem-solving skills, 
educators must develop teaching and learning strategies that use different cognitive 
components. Specific educational goals and the problems adapted for their instruction 
must in turn be designed to build specific cognitive skills. 

The investigation also suggests additional issues that seem particularly fruitful for 
future exploration. One set of issues centers around the role of justification skills in 
problem solving. This study used students’ written responses and did not use them in the 
second study for measuring Justification skills because of the lack of available 
instruments for young students. More research, using appropriate instruments, should 
explore the role of Justification skills in solving problems. Finally, future studies should 
explore whether the results found in this study generalize to other kinds of problem 
domains. Additional studies of different disciplines should examine whether the results 
are consistent with those of this study, 
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Table 1. Interconelations amon^ the Independent Variables in Problem Solving Analysis 



Variable 




1 


2 


3 


4 5 






Students (n = 1 1 


8 (1200)) 






I . Content Understanding 




... 








2. Justification Skills 




.67-' 


... 






3. Non-Cognition 




.44''' (.02) 


.34-' 






4. Knowledge of Cognition 




.27- (.20''') 


.20* 


.54''(. I7-'') 


... 


5. Regulation of Cognition 




.ly' (.13') 


.13 


.37-'' (.16-'') 


.69'' (.69'') — 


Note. ' p < .05. - 


p< .01. 


p < .001 . 


( ) numbers are the results from the Study 2. 


Table 2. Summarv of Simultaneous Reiiression Analvsi: 


s for Variables PredictiiiL^ Different Decrees of 


Structured Problem-Solving Scores 


Variables 


B 


SEB 


p 


T 


P 


Multiole Choice Well-Structured Problem-Solvinti Scores fN = 1200') 






Content Understanding 


1.22 


.03 


.73 


35.79 


.000 


Knowledge of Cognition 


1.00 


.22 


.09 


4.45 


.000 


Constant 


2.90 


.81 




3.59 


.004 


R~ = .57 (p < .000) 












Onen-Ended Well-Structured Problem-Sol viiiLi Scores (N 


= 1181 






Content Understanding 


.62 


.12 


.38 


5.04 


.000 


Justification Skills 


.34 


.06 


.43 


5.64 


.000 


Constant 


1.44 


.69 




2.1 


.038 


= .47 (p < .000) 












Near-Transfer Ill-Structured Problem-Solvinu Scores (N = 


: I 12) 






Content Understanding 


.90 


.37 


.28 


2.43 


.017 


Justification Skills 


1.11 


.28 


.46 


3.88 


.000 


Science Attitude 


.022 


.009 


.15 


2.42 


.017 


Constant 


- 2.75 


2.02 




- 1.36 


.176 


r 1 = .61 (p < .000) 












Far-Transfer Ill-Structured Problem-Solvimr Scores (N 


= 111) 






Content Understanding 


2.72 


.41 


.48 


6.57 


.000 


Justification Skills 


.49 


.12 


.32 


4.30 


.000 


Regulation of Cognition 


.26 


.10 


.20 


2.70 


.008 


Constant 


4.05 


1.87 




2.17 


.032 



RH = .48 (p < .000) 
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